Catastrophic Interference in Connectionist Networks: Can It Be Predicted, Can It Be Prevented?
نویسنده
چکیده
Catastrophic forgetting occurs when connectionist networks learn new information, and by so doing, forget all previously learned information. This workshop focused primarily on the causes of catastrophic interference, the techniques that have been developed to reduce it, the effect of these techniques on the networks' ability to generalize, and the degree to which prediction of catastrophic forgetting is possible. The speakers were Robert French, Phil Hetherington (Psychology Department, McGill University, [email protected]), and Stephan Lewandowsky (Psychology Department, University of Oklahoma, [email protected]).
منابع مشابه
Computational modeling of dynamic decision making using connectionist networks
In this research connectionist modeling of decision making has been presented. Important areas for decision making in the brain are thalamus, prefrontal cortex and Amygdala. Connectionist modeling with 3 parts representative for these 3 areas is made based the result of Iowa Gambling Task. In many researches Iowa Gambling Task is used to study emotional decision making. In these kind of decisio...
متن کاملUsing Noise to Compute Error Surfaces in Connectionist Networks: A Novel Means of Reducing Catastrophic Forgetting
In error-driven distributed feedforward networks, new information typically interferes, sometimes severely, with previously learned information. We show how noise can be used to approximate the error surface of previously learned information. By combining this approximated error surface with the error surface associated with the new information to be learned, the network's retention of previous...
متن کاملUsing Noise to Compute Error Surfaces in Connectionist Networks: A Novel Means of Reducing Catastrophic Forgetting
In error-driven distributed feedforward networks, new information typically interferes, sometimes severely,with previously learnedinformation. We show how noise can be used to approximate the error surface of previously learned information. By combining this approximated error surface with the error surface associated with the new information to be learned, the network’s retention of previously...
متن کاملCatastrophic Interference is Eliminated in Pretrained Networks
When modeling strictly sequential experimental memory tasks, such as serial list learning, connectionist networks appear to experience excessive retroactive interference, known as catastrophic interference (McCloskey & Cohen,1989; Ratcliff, 1990). The main cause of this interference is overlap among representations at the hidden unit layer (French, 1991; Hetherington,1991; Murre, 1992). This ca...
متن کاملUsing Semi-Distributed Representations to Overcome Catastrophic Forgetting in Connectionist Networks
In connectionist networks, newly-learned information destroys previously-learned information unless the network is continually retrained on the old information. This behavior, known as catastrophic forgetting, is unacceptable both for practical purposes and as a model of mind. This paper advances the claim that catastrophic forgetting is a direct consequence of the overlap of the system’s distr...
متن کامل